<!DOCTYPE html>

<html>
  <head>
    <meta charset="utf-8">
    
    <title>Setting up and using your development environment &mdash; NumPy v1.18 Manual</title>
    
    <link rel="stylesheet" type="text/css" href="../_static/css/spc-bootstrap.css">
    <link rel="stylesheet" type="text/css" href="../_static/css/spc-extend.css">
    <link rel="stylesheet" href="../_static/scipy.css" type="text/css" >
    <link rel="stylesheet" href="../_static/pygments.css" type="text/css" >
    <link rel="stylesheet" href="../_static/graphviz.css" type="text/css" >
    
    <script type="text/javascript">
      var DOCUMENTATION_OPTIONS = {
        URL_ROOT:    '../',
        VERSION:     '1.18.1',
        COLLAPSE_INDEX: false,
        FILE_SUFFIX: '.html',
        HAS_SOURCE:  false
      };
    </script>
    <script type="text/javascript" src="../_static/jquery.js"></script>
    <script type="text/javascript" src="../_static/underscore.js"></script>
    <script type="text/javascript" src="../_static/doctools.js"></script>
    <script type="text/javascript" src="../_static/language_data.js"></script>
    <script type="text/javascript" src="../_static/js/copybutton.js"></script>
    <link rel="author" title="About these documents" href="../about.html" >
    <link rel="index" title="Index" href="../genindex.html" >
    <link rel="search" title="Search" href="../search.html" >
    <link rel="top" title="NumPy v1.18 Manual" href="../index.html" >
    <link rel="up" title="Contributing to NumPy" href="index.html" >
    <link rel="next" title="Development workflow" href="development_workflow.html" >
    <link rel="prev" title="Additional Git Resources" href="gitwash/git_resources.html" > 
  </head>
  <body>
<div class="container">
  <div class="top-scipy-org-logo-header" style="background-color: #a2bae8;">
    <a href="../index.html">
      <img border=0 alt="NumPy" src="../_static/numpy_logo.png"></a>
    </div>
  </div>
</div>


    <div class="container">
      <div class="main">
        
	<div class="row-fluid">
	  <div class="span12">
	    <div class="spc-navbar">
              
    <ul class="nav nav-pills pull-left">
        <li class="active"><a href="https://numpy.org/">NumPy.org</a></li>
        <li class="active"><a href="https://numpy.org/doc">Docs</a></li>
        
        <li class="active"><a href="../index.html">NumPy v1.18 Manual</a></li>
        

          <li class="active"><a href="index.html" accesskey="U">Contributing to NumPy</a></li> 
    </ul>
              
              
    <ul class="nav nav-pills pull-right">
      <li class="active">
        <a href="../genindex.html" title="General Index"
           accesskey="I">index</a>
      </li>
      <li class="active">
        <a href="development_workflow.html" title="Development workflow"
           accesskey="N">next</a>
      </li>
      <li class="active">
        <a href="gitwash/git_resources.html" title="Additional Git Resources"
           accesskey="P">previous</a>
      </li>
    </ul>
              
	    </div>
	  </div>
	</div>
        

	<div class="row-fluid">
      <div class="spc-rightsidebar span3">
        <div class="sphinxsidebarwrapper">
  <h3><a href="../contents.html">Table of Contents</a></h3>
  <ul>
<li><a class="reference internal" href="#">Setting up and using your development environment</a><ul>
<li><a class="reference internal" href="#recommended-development-setup">Recommended development setup</a></li>
<li><a class="reference internal" href="#testing-builds">Testing builds</a></li>
<li><a class="reference internal" href="#building-in-place">Building in-place</a></li>
<li><a class="reference internal" href="#other-build-options">Other build options</a></li>
<li><a class="reference internal" href="#using-virtualenvs">Using virtualenvs</a></li>
<li><a class="reference internal" href="#running-tests">Running tests</a></li>
<li><a class="reference internal" href="#rebuilding-cleaning-the-workspace">Rebuilding &amp; cleaning the workspace</a></li>
<li><a class="reference internal" href="#debugging">Debugging</a></li>
<li><a class="reference internal" href="#understanding-the-code-getting-started">Understanding the code &amp; getting started</a></li>
</ul>
</li>
</ul>

  <h4>Previous topic</h4>
  <p class="topless"><a href="gitwash/git_resources.html"
                        title="previous chapter">Additional Git Resources</a></p>
  <h4>Next topic</h4>
  <p class="topless"><a href="development_workflow.html"
                        title="next chapter">Development workflow</a></p>
<div id="searchbox" style="display: none" role="search">
  <h4>Quick search</h4>
    <div>
    <form class="search" action="../search.html" method="get">
      <input type="text" style="width: inherit;" name="q" />
      <input type="submit" value="search" />
      <input type="hidden" name="check_keywords" value="yes" />
      <input type="hidden" name="area" value="default" />
    </form>
    </div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
        </div>
      </div>
          <div class="span9">
            
        <div class="bodywrapper">
          <div class="body" id="spc-section-body">
            
  <div class="section" id="setting-up-and-using-your-development-environment">
<span id="development-environment"></span><h1>Setting up and using your development environment<a class="headerlink" href="#setting-up-and-using-your-development-environment" title="Permalink to this headline">¶</a></h1>
<div class="section" id="recommended-development-setup">
<span id="id1"></span><h2>Recommended development setup<a class="headerlink" href="#recommended-development-setup" title="Permalink to this headline">¶</a></h2>
<p>Since NumPy contains parts written in C and Cython that need to be
compiled before use, make sure you have the necessary compilers and Python
development headers installed - see <a class="reference internal" href="../user/building.html#building-from-source"><span class="std std-ref">Building from source</span></a>. Building
NumPy as of version <code class="docutils literal notranslate"><span class="pre">1.17</span></code> requires a C99 compliant compiler.</p>
<p>Having compiled code also means that importing NumPy from the development
sources needs some additional steps, which are explained below.  For the rest
of this chapter we assume that you have set up your git repo as described in
<a class="reference internal" href="gitwash/index.html#using-git"><span class="std std-ref">Git for development</span></a>.</p>
</div>
<div class="section" id="testing-builds">
<span id="id2"></span><h2>Testing builds<a class="headerlink" href="#testing-builds" title="Permalink to this headline">¶</a></h2>
<p>To build the development version of NumPy and run tests, spawn
interactive shells with the Python import paths properly set up etc.,
do one of:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ python runtests.py -v
$ python runtests.py -v -s random
$ python runtests.py -v -t numpy/core/tests/test_nditer.py::test_iter_c_order
$ python runtests.py --ipython
$ python runtests.py --python somescript.py
$ python runtests.py --bench
$ python runtests.py -g -m full
</pre></div>
</div>
<p>This builds NumPy first, so the first time it may take a few minutes.  If
you specify <code class="docutils literal notranslate"><span class="pre">-n</span></code>, the tests are run against the version of NumPy (if
any) found on current PYTHONPATH.</p>
<p>When specifying a target using <code class="docutils literal notranslate"><span class="pre">-s</span></code>, <code class="docutils literal notranslate"><span class="pre">-t</span></code>, or <code class="docutils literal notranslate"><span class="pre">--python</span></code>, additional
arguments may be forwarded to the target embedded by <code class="docutils literal notranslate"><span class="pre">runtests.py</span></code> by passing
the extra arguments after a bare <code class="docutils literal notranslate"><span class="pre">--</span></code>. For example, to run a test method with
the <code class="docutils literal notranslate"><span class="pre">--pdb</span></code> flag forwarded to the target, run the following:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ python runtests.py -t numpy/tests/test_scripts.py:test_f2py -- --pdb
</pre></div>
</div>
<p>When using pytest as a target (the default), you can
<a class="reference external" href="https://docs.pytest.org/en/latest/usage.html#specifying-tests-selecting-tests">match test names using python operators</a> by passing the <code class="docutils literal notranslate"><span class="pre">-k</span></code> argument to pytest:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ python runtests.py -v -t numpy/core/tests/test_multiarray.py -- -k &quot;MatMul and not vector&quot;
</pre></div>
</div>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>Remember that all tests of NumPy should pass before committing your changes.</p>
</div>
<p>Using <code class="docutils literal notranslate"><span class="pre">runtests.py</span></code> is the recommended approach to running tests.
There are also a number of alternatives to it, for example in-place
build or installing to a virtualenv. See the FAQ below for details.</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>Some of the tests in the test suite require a large amount of
memory, and are skipped if your system does not have enough.</p>
<p>To override the automatic detection of available memory, set the
environment variable <code class="docutils literal notranslate"><span class="pre">NPY_AVAILABLE_MEM</span></code>, for example
<code class="docutils literal notranslate"><span class="pre">NPY_AVAILABLE_MEM=32GB</span></code>, or using pytest <code class="docutils literal notranslate"><span class="pre">--available-memory=32GB</span></code>
target option.</p>
</div>
</div>
<div class="section" id="building-in-place">
<h2>Building in-place<a class="headerlink" href="#building-in-place" title="Permalink to this headline">¶</a></h2>
<p>For development, you can set up an in-place build so that changes made to
<code class="docutils literal notranslate"><span class="pre">.py</span></code> files have effect without rebuild. First, run:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ python setup.py build_ext -i
</pre></div>
</div>
<p>This allows you to import the in-place built NumPy <em>from the repo base
directory only</em>.  If you want the in-place build to be visible outside that
base dir, you need to point your <code class="docutils literal notranslate"><span class="pre">PYTHONPATH</span></code> environment variable to this
directory.  Some IDEs (<a class="reference external" href="https://www.spyder-ide.org/">Spyder</a> for example) have utilities to manage
<code class="docutils literal notranslate"><span class="pre">PYTHONPATH</span></code>.  On Linux and OSX, you can run the command:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ export PYTHONPATH=$PWD
</pre></div>
</div>
<p>and on Windows:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ set PYTHONPATH=/path/to/numpy
</pre></div>
</div>
<p>Now editing a Python source file in NumPy allows you to immediately
test and use your changes (in <code class="docutils literal notranslate"><span class="pre">.py</span></code> files), by simply restarting the
interpreter.</p>
<p>Note that another way to do an inplace build visible outside the repo base dir
is with <code class="docutils literal notranslate"><span class="pre">python</span> <span class="pre">setup.py</span> <span class="pre">develop</span></code>.  Instead of adjusting <code class="docutils literal notranslate"><span class="pre">PYTHONPATH</span></code>, this
installs a <code class="docutils literal notranslate"><span class="pre">.egg-link</span></code> file into your site-packages as well as adjusts the
<code class="docutils literal notranslate"><span class="pre">easy-install.pth</span></code> there, so its a more permanent (and magical) operation.</p>
</div>
<div class="section" id="other-build-options">
<h2>Other build options<a class="headerlink" href="#other-build-options" title="Permalink to this headline">¶</a></h2>
<p>Build options can be discovered by running any of:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ python setup.py --help
$ python setup.py --help-commands
</pre></div>
</div>
<p>It’s possible to do a parallel build with <code class="docutils literal notranslate"><span class="pre">numpy.distutils</span></code> with the <code class="docutils literal notranslate"><span class="pre">-j</span></code> option;
see <a class="reference internal" href="../user/building.html#parallel-builds"><span class="std std-ref">Parallel builds</span></a> for more details.</p>
<p>A similar approach to in-place builds and use of <code class="docutils literal notranslate"><span class="pre">PYTHONPATH</span></code> but outside the
source tree is to use:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ pip install . --prefix /some/owned/folder
$ export PYTHONPATH=/some/owned/folder/lib/python3.4/site-packages
</pre></div>
</div>
<p>NumPy uses a series of tests to probe the compiler and libc libraries for
funtions. The results are stored in <code class="docutils literal notranslate"><span class="pre">_numpyconfig.h</span></code> and <code class="docutils literal notranslate"><span class="pre">config.h</span></code> files
using <code class="docutils literal notranslate"><span class="pre">HAVE_XXX</span></code> definitions. These tests are run during the <code class="docutils literal notranslate"><span class="pre">build_src</span></code>
phase of the <code class="docutils literal notranslate"><span class="pre">_multiarray_umath</span></code> module in the <code class="docutils literal notranslate"><span class="pre">generate_config_h</span></code> and
<code class="docutils literal notranslate"><span class="pre">generate_numpyconfig_h</span></code> functions. Since the output of these calls includes
many compiler warnings and errors, by default it is run quietly. If you wish
to see this output, you can run the <code class="docutils literal notranslate"><span class="pre">build_src</span></code> stage verbosely:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ python build build_src -v
</pre></div>
</div>
</div>
<div class="section" id="using-virtualenvs">
<h2>Using virtualenvs<a class="headerlink" href="#using-virtualenvs" title="Permalink to this headline">¶</a></h2>
<p>A frequently asked question is “How do I set up a development version of NumPy
in parallel to a released version that I use to do my job/research?”.</p>
<p>One simple way to achieve this is to install the released version in
site-packages, by using a binary installer or pip for example, and set
up the development version in a virtualenv.  First install
<a class="reference external" href="http://www.virtualenv.org/">virtualenv</a> (optionally use <a class="reference external" href="http://www.doughellmann.com/projects/virtualenvwrapper/">virtualenvwrapper</a>), then create your
virtualenv (named numpy-dev here) with:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ virtualenv numpy-dev
</pre></div>
</div>
<p>Now, whenever you want to switch to the virtual environment, you can use the
command <code class="docutils literal notranslate"><span class="pre">source</span> <span class="pre">numpy-dev/bin/activate</span></code>, and <code class="docutils literal notranslate"><span class="pre">deactivate</span></code> to exit from the
virtual environment and back to your previous shell.</p>
</div>
<div class="section" id="running-tests">
<h2>Running tests<a class="headerlink" href="#running-tests" title="Permalink to this headline">¶</a></h2>
<p>Besides using <code class="docutils literal notranslate"><span class="pre">runtests.py</span></code>, there are various ways to run the tests.  Inside
the interpreter, tests can be run like this:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">&gt;&gt;&gt; </span><span class="n">np</span><span class="o">.</span><span class="n">test</span><span class="p">()</span>  
<span class="gp">&gt;&gt;&gt; </span><span class="n">np</span><span class="o">.</span><span class="n">test</span><span class="p">(</span><span class="s1">&#39;full&#39;</span><span class="p">)</span>   <span class="c1"># Also run tests marked as slow</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">np</span><span class="o">.</span><span class="n">test</span><span class="p">(</span><span class="s1">&#39;full&#39;</span><span class="p">,</span> <span class="n">verbose</span><span class="o">=</span><span class="mi">2</span><span class="p">)</span>   <span class="c1"># Additionally print test name/file</span>

<span class="go">An example of a successful test :</span>
<span class="go">``4686 passed, 362 skipped, 9 xfailed, 5 warnings in 213.99 seconds``</span>
</pre></div>
</div>
<p>Or a similar way from the command line:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ python -c &quot;import numpy as np; np.test()&quot;
</pre></div>
</div>
<p>Tests can also be run with <code class="docutils literal notranslate"><span class="pre">pytest</span> <span class="pre">numpy</span></code>, however then the NumPy-specific
plugin is not found which causes strange side effects</p>
<p>Running individual test files can be useful; it’s much faster than running the
whole test suite or that of a whole module (example: <code class="docutils literal notranslate"><span class="pre">np.random.test()</span></code>).
This can be done with:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ python path_to_testfile/test_file.py
</pre></div>
</div>
<p>That also takes extra arguments, like <code class="docutils literal notranslate"><span class="pre">--pdb</span></code> which drops you into the Python
debugger when a test fails or an exception is raised.</p>
<p>Running tests with <a class="reference external" href="https://tox.readthedocs.io/">tox</a> is also supported.  For example, to build NumPy and
run the test suite with Python 3.7, use:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ tox -e py37
</pre></div>
</div>
<p>For more extensive information, see <a class="reference internal" href="../reference/testing.html#testing-guidelines"><span class="std std-ref">Testing Guidelines</span></a></p>
<p><em>Note: do not run the tests from the root directory of your numpy git repo without ``runtests.py``,
that will result in strange test errors.</em></p>
</div>
<div class="section" id="rebuilding-cleaning-the-workspace">
<h2>Rebuilding &amp; cleaning the workspace<a class="headerlink" href="#rebuilding-cleaning-the-workspace" title="Permalink to this headline">¶</a></h2>
<p>Rebuilding NumPy after making changes to compiled code can be done with the
same build command as you used previously - only the changed files will be
re-built.  Doing a full build, which sometimes is necessary, requires cleaning
the workspace first.  The standard way of doing this is (<em>note: deletes any
uncommitted files!</em>):</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ git clean -xdf
</pre></div>
</div>
<p>When you want to discard all changes and go back to the last commit in the
repo, use one of:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ git checkout .
$ git reset --hard
</pre></div>
</div>
</div>
<div class="section" id="debugging">
<h2>Debugging<a class="headerlink" href="#debugging" title="Permalink to this headline">¶</a></h2>
<p>Another frequently asked question is “How do I debug C code inside NumPy?”.
The easiest way to do this is to first write a Python script that invokes the C
code whose execution you want to debug. For instance <code class="docutils literal notranslate"><span class="pre">mytest.py</span></code>:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">numpy</span> <span class="kn">import</span> <span class="n">linspace</span>
<span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">arange</span><span class="p">(</span><span class="mi">5</span><span class="p">)</span>
<span class="n">np</span><span class="o">.</span><span class="n">empty_like</span><span class="p">(</span><span class="n">x</span><span class="p">)</span>
</pre></div>
</div>
<p>Now, you can run:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ gdb --args python runtests.py -g --python mytest.py
</pre></div>
</div>
<p>And then in the debugger:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">(</span><span class="n">gdb</span><span class="p">)</span> <span class="k">break</span> <span class="n">array_empty_like</span>
<span class="p">(</span><span class="n">gdb</span><span class="p">)</span> <span class="n">run</span>
</pre></div>
</div>
<p>The execution will now stop at the corresponding C function and you can step
through it as usual.  With the Python extensions for gdb installed (often the
default on Linux), a number of useful Python-specific commands are available.
For example to see where in the Python code you are, use <code class="docutils literal notranslate"><span class="pre">py-list</span></code>.  For more
details, see <a class="reference external" href="https://wiki.python.org/moin/DebuggingWithGdb">DebuggingWithGdb</a>.</p>
<p>Instead of plain <code class="docutils literal notranslate"><span class="pre">gdb</span></code> you can of course use your favourite
alternative debugger; run it on the python binary with arguments
<code class="docutils literal notranslate"><span class="pre">runtests.py</span> <span class="pre">-g</span> <span class="pre">--python</span> <span class="pre">mytest.py</span></code>.</p>
<p>Building NumPy with a Python built with debug support (on Linux distributions
typically packaged as <code class="docutils literal notranslate"><span class="pre">python-dbg</span></code>) is highly recommended.</p>
</div>
<div class="section" id="understanding-the-code-getting-started">
<h2>Understanding the code &amp; getting started<a class="headerlink" href="#understanding-the-code-getting-started" title="Permalink to this headline">¶</a></h2>
<p>The best strategy to better understand the code base is to pick something you
want to change and start reading the code to figure out how it works. When in
doubt, you can ask questions on the mailing list. It is perfectly okay if your
pull requests aren’t perfect, the community is always happy to help. As a
volunteer project, things do sometimes get dropped and it’s totally fine to
ping us if something has sat without a response for about two to four weeks.</p>
<p>So go ahead and pick something that annoys or confuses you about numpy,
experiment with the code, hang around for discussions or go through the
reference documents to try to fix it. Things will fall in place and soon
you’ll have a pretty good understanding of the project as a whole. Good Luck!</p>
</div>
</div>


          </div>
        </div>
          </div>
        </div>
      </div>
    </div>

    <div class="container container-navbar-bottom">
      <div class="spc-navbar">
        
      </div>
    </div>
    <div class="container">
    <div class="footer">
    <div class="row-fluid">
    <ul class="inline pull-left">
      <li>
        &copy; Copyright 2008-2019, The SciPy community.
      </li>
      <li>
      Last updated on Feb 20, 2020.
      </li>
      <li>
      Created using <a href="http://sphinx.pocoo.org/">Sphinx</a> 2.4.2.
      </li>
    </ul>
    </div>
    </div>
    </div>
  </body>
</html>